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Examiner' s Detailed Office Action 



1. This Office is responsive to the communication for 
application 10/711,191, filed Novemeber 16, 2007. 

2. Claims 1-19 have been examined. 

3. After the previous examination, claims 1-19 stood rejected. 



4. The following is a quotation of the first paragraph of 35 
U.S.C. 112: 

The specification shall contain a written description of the invention, 
and of the manner and process of making and using it, in such full, 
clear, concise, and exact terms as to enable any person skilled in the 

art to which it pertains, or with which it is most nearly connected, to 
make and use the same and shall set forth the best mode contemplated by 
the inventor of carrying out his invention. 

5. Claims 1 and 4-19 are rejected under 35 U.S.C. 112, first 
paragraph. Specifically, if the application fails as a matter of 
fact to satisfy 35 U.S.C. § 101, then the application also fails 
as a matter of law to enable one of ordinary skill in the art to 



Claim Rejections 



- 35 use § 112, 1 



St 
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use the invention under 35 U.S.C. § 112.; In re Kirk, 376 F.2d 
936, 942, 153 USPQ 48, 53 (CCPA 1967) MPEP 2107.01 (IV). 



Claim Rejections - 35 USC § 101 



6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, 
manufacture, or composition of matter, or any new and useful improvement 
thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 



7. Claims 1 and 4-12 are rejected under 35 U.S.C. 101 because 
the claimed invention is directed to non-statutory subject 
matter: mathematical abstraction and/or algorithm. Independent 
claim 1 recites claim a "neural network" comprising a plurality 
of nodes and directed edges and a training process. The final 
result recited is "summing the first summand and the additional 
summands, wherein, in estimating said derivative, paths from 
said second node to said output node that involve said third 
plurality of directed edges are not considered". Examiner 
considers a list of nodes and edges to be no more that a 
recitation of a graph model. Examiner considers the training 
process to be no more than an algorithm. Therefore claim 1 
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recites no more than the 101 judicial exceptions of mathematical 
abstraction and algorithm and is considered non-statutory under 
35 U.S.C. 101. Claims 4-12 provide detailed algorithm 
limitation to claim 1 and thus fail to cure the deficiency of 
claim 1. Therefore, claims 1 and 4-12 are non-statutory under 35 
U.S.C. 101. 

8. Claims 13-19 are rejected under 35 U.S.C. 101 because the 
claimed invention is directed to non-statutory subject matter: 
algorithm. Independent claim 13 recites a "method of training a 
neural network" with the graph structure of claim 1 . The final 
result of the training is "summing the first summand and the 
additional summands, wherein, in estimating said derivative, 
paths from said second node to said output node that involve 
said third plurality of directed edges are not considered" which 
is considered to be a numerical value representing no credible 
or specific real-world result. Claims 14-19 provide detailed 
algorithm limitation to claim 13 and thus fail to cure the 
deficiency of claim 13. Therefore, claims 13-19 are non- 
statutory under 35 U.S.C. 101. 
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Claim Rejections - 35 USC § 102 

9. The following is a quotation of the appropriate paragraphs 
of 35 U.S.C. 102 that form the basis for the rejections under 
this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in 
this or a foreign country or in public use or on sale in this country, 
more than one year prior to the date of application for patent in the 
United States . 

10. Claims 1, 4-7, 11, 13, 14, and 18 are rejected under 35 
U.S.C. 102(b) as being anticipated by Fahlman et al. (Fahlman) , 
■^^The Cascade-Correlation Learning Architecture", 1990. 

Regarding claim 1. Fahlman teaches a neural network comprising: 
a first node (see p. 4, Fig. 1, Examiner "Hidden Unit 1" to 

be a first node.) ; 

a second node adapted to receive and process signals from 

said first node (see p. 4, Fig. 1, Examiner interprets "Hidden 

Unit 2" to be a second node adapted to receive and process 

signals from said first node.); 

a first directed edge between said first node and said 

second node for transmitting signals from said first node to 
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said second node, wherein said first directed edge is 
characterized by a first weight (see p. 4, Fig. 1, Examiner 
interprets the output path from "Hidden Unit 1" to intersecting 
the vertical path to "Hidden Unit 2" be a first directed edge 
between said first node and said second node for transmitting 
signals from said first node to said second node. Examiner 
interprets the "X connection"' between "Hidden Unit 1" and 
"Hidden Unit 2" to be a first weight. ); 

an output node adapted to receive and process signals from 
said second node (see p. 4, Fig. 1, Examiner interprets the left 
most "Outputs" to be an output node adapted to receive and 
process signals from said second node.); 

a second directed edge between said second node and said 
output node for transmitting signals from said second node to 
said output node, wherein said second directed edge is 
characterized by a second weight (see p. 4, Fig. 1, Examiner 
interprets the output path from "Hidden Unit 2" intersecting the 
vertical path to the left most "Outputs" to be a second 
directed edge between said second node and said output node for 
transmitting signals from said second node to said output node, 
wherein said second directed edge is characterized by a second 
weight. Examiner interprets the "X connection" between "Hidden 
Unit 1" and the left most "Outputs" to be a second weight.); 
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a plurality of additional nodes between said second node 
and said output node (see p. 4, Fig. 1, Examiner interprets 
"Hidden Unit 2" to be a subset of a plurality of additional 
nodes "hidden units" (see Abstract) between said second node and 
said output node.); 

a first plurality of directed edges coupling said second 
node to said plurality of additional nodes (see p. 4, Fig. 1, 
Examiner interprets the output path from "Hidden Unit 2" 
intersecting the vertical paths to each of said plurality of 
additional nodes to be a first plurality of directed edges 
coupling said second node to said plurality of additional 
nodes . ) ; 

a second plurality of directed edges coupling said 
plurality of additional nodes to said output node (see p. 4, 
Fig. 1, Examiner interprets the output path from each of said 
plurality of additional nodes intersecting the vertical path to 
the left most "Outputs" to be a second plurality of directed 
edges coupling said plurality of additional nodes to said output 
node. ) ; 

a third plurality of directed edges coupling signals from 
nodes among said plurality of additional nodes to other nodes 
among said plurality of additional nodes that are closer to said 
output node (see p. 3 and p. 4, Fig. 1, §2. Description of 
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Cascade-Correlation, "We add hidden units to the network one by 
one. Each new hidden unit receives a connection from each of the 
network' s original inputs and also from every pre-existing 
hidden unit. The hidden unit's input weights are frozen at the 
time the unit is added to the net; only the output connections 
are trained repeatedly.". Examiner interprets the "output 
connections" of "hidden units"' to the "input connections" of 
"hidden units" be a third plurality of directed edges coupling 
signals from nodes among said plurality of additional nodes to 
other nodes among said plurality of additional nodes that are 
closer to said output node.); 

wherein, said first weight has a value that is determined 
by a process of training said neural network (see p. 4, Fig. 1, 
Examiner interprets a "X connetions" between units to be a 
weight that has a value that is determined by a process of 
training said neural network .) that comprises: 

estimating a derivative of a summed input to said 

output node with respect to said first weight by: 

multiplying a signal output by said first node by a 

value of a derivative of a transfer function of said second 

node that obtains when training data is applied to said 

neural network to obtain a first factor; 
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multiplying said first factor by said second weight to 
compute a first summand; 

for each particular node of the plurality of 
additional nodes between said second node and said 
output node, computing an additional summand by 
multiplying together the first factor, a weight 
characterizing one of the first plurality of directed 
edges that couples the second node to the particular 
node, a weight characterizing one of the second 
plurality of directed edges that couples the 
particular node to the output node, and a value of a 
transfer function of the particular node; and 

summing the first summand and the additional 
summands, wherein, in estimating said derivative, 
paths from said second node to said output node that 
involve said third plurality of directed edges are not 
considered (see p. 5, "After computing 5S/5wi for each 
incoming connection, we can perform a gradient ascent 
to maximize S. Once again we are training only a 
single layer of weights. Once again we use the 
quickprop update rule for faster convergence.". 
Examiner interprets gradient ascent using guickprop to 
comprise the above training steps.). 
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Regarding claim 13. Fahlman teaches a method of training a 
neural network that comprises: 

a first node (see p. 4, Fig. 1, Examiner "Hidden Unit 1" to 
be a first node.); 

a second node adapted to receive and process signals from 
said first node (see p. 4, Fig. 1, Examiner interprets "Hidden 
Unit 2" to be a second node adapted to receive and process 
signals from said first node.); 

a first directed edge between said first node and said 
second node for transmitting signals from said first node to 
said second node, wherein said first directed edge is 
characterized by a first weight (see p. 4, Fig. 1, Examiner 
interprets the output path from "Hidden Unit 1" to intersecting 
the vertical path to "Hidden Unit 2" be a first directed edge 
between said first node and said second node for transmitting 
signals from said first node to said second node. Examiner 
interprets the "X connection" between "Hidden Unit 1" and 
"Hidden Unit 2" to be a first weight.); 

an output node adapted to receive and process signals from 
said second node (see p. 4, Fig. 1, Examiner interprets the left 
most "Outputs'' to be an output node adapted to receive and 
process signals from said second node.); 
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a second directed edge between said second node and said 
output node for transmitting signals from said second node to 
said output node, wherein said second directed edge is 
characterized by a second weight (see p. 4, Fig. 1, Examiner 
interprets the output path from "Hidden Unit 2" intersecting the 
vertical path to the left most "Outputs" to be a second 
directed edge between said second node and said output node for 
transmitting signals from said second node to said output node, 
wherein said second directed edge is characterized by a second 
weight. Examiner interprets the "X connection" between "Hidden 
Unit 1" and the left most "Outputs" to be a second weight.); 

a plurality of additional nodes between said second node 
and said output node (see p. 4, Fig. 1, Examiner interprets 
"Hidden Unit 2" to be a subset of a plurality of additional 
nodes "hidden units" (see Abstract) between said second node and 
said output node.); 

a first plurality of directed edges coupling said second 
node to said plurality of additional nodes; a second plurality 
of directed edges coupling said plurality of additional nodes to 
said output node (see p. 4, Fig. 1, Examiner interprets the 
output path from "Hidden Unit 2" intersecting the vertical paths 
to each of said plurality of additional nodes to be a first 
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plurality of directed edges coupling said second node to said 
plurality of additional nodes.); 

a third plurality of directed edges coupling signals from 
nodes among said plurality of additional nodes to other nodes 
among said plurality of additional nodes that are closer to said 
output node (see p. 3 and p. 4, Fig. 1, §2. Description of 
Cascade-Correlation, "We add hidden units to the network one by 
one. Each new hidden unit receives a connection from each of the 
network' s original inputs and also from every pre-existing 
hidden unit. The hidden unit's input weights are frozen at the 
time the unit is added to the net; only the output connections 
are trained repeatedly.". Examiner interprets the "output 
connections" of "hidden units" to the "input connections" of 
"hidden units" be a third plurality of directed edges coupling 
signals from nodes among said plurality of additional nodes to 
other nodes among said plurality of additional nodes that are 
closer to said output node.); 

the method comprising: 

estimating a derivative of a summed input to said 

output node with respect to said first weight by: 

multiplying a signal output by said first node by a 

value of a derivative of a transfer function of said second 
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node that obtains when training data is applied to said 
neural network to obtain a first factor; 

multiplying said first factor by said second weight to 
compute a first summand; 

for each particular node of the plurality of additional 
nodes between said second node and said output node, computing 
an additional summand by multiplying together the first factor, 
a weight characterizing one of the first plurality of directed 
edges that couples the second node to the particular node, a 
weight characterizing one of the second plurality of directed 
edges that couples the particular node to the output node, and a 
value of a transfer function of the particular node; and 

summing the first summand and the additional summands, 
wherein, in estimating said derivative, paths from said 
second node to said output node that involve said third 
plurality of directed edges are not considered (see p. 5, 
■^^After computing 5S/5wi for each incoming connection, we can 
perform a gradient ascent to maximize S. Once again we are 
training only a single layer of weights. Once again we use 
the quickprop update rule for faster convergence.". 
Examiner interprets gradient ascent using quickprop to 
comprise the above training steps.). 
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Regarding claim 4 . Fahlman teaches the neural network according 
to claim 1 wherein said first node comprises an input of said 
neural network (see p. 4, Fig. 1, Examiner interprets any 
"Inputs" to be a first node.) . 

Regarding claim 5. Fahlman teaches the neural network according 
to claim 1 wherein said first node comprises a hidden processing 
node of said neural network (see p. 4, Fig. 1, Examiner "Hidden 
Unit 1" to he a first node.) . 

Regarding claim 6. Fahlman teaches the neural network according 
to claim 1 wherein: said plurality of additional nodes include 
sigmoid transfer functions (see p. 3, "In the experiments we 
have run so far, we use a symmetric sigmoidal activation 
function (hyperbolic tangent) whose output range is -1.0 to 
+ 1.0.") . 

Regarding claims 7 and 14. Fahlman teaches the neural network 
according to claim 1 and method of training the neural network 
according to claim 13 wherein said process of training said 
neural network comprises: 

(a) applying training data to said neural network, whereby 
said summed input is generated at said output node; 
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(b) computing a value of a derivative of an objective 
function that depends on said derivative of said summed input to 
said output node with respect to said first weight; 

(c) processing said derivative of said objective function 
with an optimization algorithm that uses derivative information; 
and 

(d) repeating (a) -(c) until a stopping condition is 
satisfied (see pp. 3-5, "We run a number of passes over the 
examples of the training set, adjusting the candidate unit's 
input weights after each pass... When S stops improving, we 
install the new candidate as a unit in the active network, 
freeze its input weights, and continue the cycle as described 
above.". Examiner interprets gradient ascent using quickprop to 
comprise steps (a) -(d).). 

Regarding claims 11 and 18. Fahlman teaches the neural network 
according to claim 10 and method of training the neural network 
according to claim 17 wherein: the objective function is a 
function of a difference an actual output of said neural network 
that depends on said summed input to said output node and an 
expected output (see p. 5, Examiner interprets S to be an 
obj ective function to be maximized . Examiner interprets Vp to be 
a candidate unit's value which depends on a summed input for 
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pattern p. Examiner interprets Vbar to be the average or 
expected output of the candidate unit over all patterns, p.); 
and the objective function is a continuously dif f erentiable 
function of a measure of near zero weights (see p. 5, Examiner 
interprets 5S/5w± to he a continuously dif f erentiable function of 
a measure of near zero weights as Gq is just a sign function and 
fp is the first derivative of a symmetric sigmoidal activation 
function (hyperbolic tangent) or 1/cosh^ (x) =1/ (e^ + e~^/2) which 
approaches 2 when the summed input, x, approaches zero for near 
zero weights . ) . 



11. The following is a quotation of 35 U.S.C. 103(a) which forms 
the basis for all obviousness rejections set forth in this 

Office action: 

(a) A patent may not be obtained though the invention is not 
identically disclosed or described as set forth in section 102 of this 
title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person 



having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the 
invention was made . 



Claim Rejections 



- 35 use § 103 



12. Claim 2 is rejected under 35 U.S.C. 103(a) as being 
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unpatentable over Fahlman in view of Mashiko (USPN: 5,202,956). 

Regarding claim 2 . Fahlman teaches the neural network according 
to claim 1. Fahlman does not teach said first directed edge, 
said second directed edge, said first plurality of directed 
edges and said second plurality of directed edges comprise one 
or more amplifying circuits. Mashiko does teach said first 
directed edge, said second directed edge, said first plurality 
of directed edges and said second plurality of directed edges 
comprise one or more amplifying circuits (see Fig. 1 and col. 4, 
lines 51-54, Examiner interprets the "the converting portion in 
FIG. 1" to be a directed (output) edge.). It would have been 
obvious at the time the invention was made to persons having 
ordinary skill in the art to combine Fahlman with Mashiko in 
order to provide multi-valued expression of coupling strength 
with fewer coupling elements. 



Claim Rejections - 35 USC §103 



13. Claim 3 is rejected under 35 U.S.C. 103(a) as being 
unpatentable over Fahlman in view of Smyth (USPN: 6,092,058). 
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Regarding claim 3. Fahlman teaches the neural network according 
to claim 1. Fahlman does not teach said first directed edge, 
said second directed edge, said first plurality of directed 
edges, and said second plurality of directed edges comprise one 
or more attenuating circuits. Smyth does teach said first 
directed edge, said second directed edge, said first plurality 
of directed edges, and said second plurality of directed edges 
comprise one or more attenuating circuits (see Fig. 5 and col. 4 
lines 2 9-38, Examiner interprets "the attenuations of the 
circuit model are the linear expansion coefficient weights" to 
each connection (edge) in the cerebrum model is an attenuating 
circuit.). It would have been obvious at the time the invention 
was made to persons having ordinary skill in the art to combine 
Fahlman with Smyth in order to attain high levels of decision 
classification accuracy by combining a parametric model of 
cerebral potential with advanced techniques drawn from numerical 
analysis, artificial intelligence, and nonlinear regression 
analysis . 

14. Claims 8 and 15 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Fahlman in view of Watrous , "Learning 
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Algorithms for Connectionist Networks: Applied Gradient Methods 
of Nonlinear Optimization", 1988. 



Regarding claims 8 and 15. Fahlman teaches the neural network 
according to claim 7 and method of training the neural network 
according to claim 14. Fahlman does not teach the said process 
of training said neural network, processing said derivative of 
said objective function comprises: using a nonlinear 
optimization algorithm selected from the group consisting of the 
steepest descent method, the conjugate gradient method, and the 
Broyden-Fletcher-Goldf arb-Shanno method. However, Watrous does 
teach the said process of training said neural network, 
processing said derivative of said objective function comprises: 
using a nonlinear optimization algorithm selected from the group 
consisting of the steepest descent method, the conjugate 
gradient method, and the Broyden-Fletcher-Goldf arb-Shanno method 
(see Abstract) . It would have been obvious at the time the 
invention was made to persons having ordinary skill in the art 
to combine Fahlman with Watrous in order to utilize additional 
information about the shape of the weight space to obtain 
dramatically faster convergence. 
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Response to Arguments 

15. Applicant's arguments filed November 16, 2007 have been 
fully considered but they are not persuasive. 

Rejection of Claims 1-12 and 13-19 Under 35 U.S.C. §101 



Applciants argue: 



One assertion made in the Office Action is that the 
applicant's claims are directed to a "mathematical 
abstraction". Set Theory or the Fundamental Theorem of 
Calculus may be considered to be "mathematical 
abstractions" but an apparatus for processing signals such 
as a neural network is not a "mathematical abstraction". 



Examiner responds: 



Examiner disagrees. A neural network, as claimed by 
Applicants, in independent claim 1, is nothing more than a 
listing of nodes, edges, weights, and arithmetic 
operations. Nodes, edges, weights are clearly recognizable 
as entities from the theory of weighted directed graphs. 
The arithmetic operations over the nodes, edges, and 
weights are well known in simple vector-matrix arithmetic. 



Applciants argue: 



As indicated in the applicant's specification neural 
networks, including the applicant's claimed neural network 
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can be implemented using analog or digital electronics and 
in either form the claimed neural network is not a 
"mathematical abstraction". (It is noted that the 
applicant's independent claims are not so narrow as to be 
restricted to analog or digital electronics. One of 
ordinary skill in the art can implement the applicant's 
claimed neural networks using analog or digital circuitry.) 

Examiner responds: 

Examiner reminds Applicants, that it is not what Applicants 
invention can be, but what it is claimed to be that 
Applicants seek to exclude others from using. Applicants 
do not claim any type of implementation (not even software 
per se) . Clearly, Applicants seek to exclude others from 
applying certain vector-matrix arithmetic operations to a 
particular graph structure. This is considered to be non- 
statutory as nothing other than the 101 judicial exceptions 
of mathematical abstraction and arithmetic algorithm (i.e., 
the operations in the claimed order) are recited in the 
claims . 



Applciants argue: 

The utility of neural networks for processing signals is 
well known to workers in various advanced fields of 
electrical engineering. 

Examiner responds: 

Examiner agrees. However Applicants recite no physical 

transformation or useful, concrete, and tangible final 
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result in independent claims 1, 4-12, or 13-19. Therefore, 
Applicants claims a considered to have no utility. 

Applciants argue: 

In the case of the neural networks claimed in claims 1-12, 
such an algorithm is not abstract and detached from 
practical utility, rather it is part of a signal processor 
that as such has practical utility. 

Examiner responds: 

Applicants have not claimed a signal processor. Examiner 
reminds Applicants that a signal processor, per se, is 
considered to be no more than a mathematical entity that 
performs the analysis, interpretation, and manipulation of 
signals (i.e., a functional). 

Applciants argue: 

In rejecting the applicants claims under 35 U.S.C. §101 the 
office action also asserted that the claims 1-12 were drawn 
to a "computer related manufacture". 

Examiner responds: 

Applicants' argument is moot, based on new grounds of 

rejection under 35 U.S.C. §101. 

Applciants argue: 

Despite the lack of utility for such components or sub- 
systems, in isolation, such components and sub-systems are 
statutory under 35 U.S.C. §101. The standards of 35 U.S.C. 
§101 and 35 U.S.C. §112 are such that one can obtain a 
patent on a component or sub-system without describing in 
the patent specification the details of larger systems in 
which the component or sub-system is used. A patent on a 
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carburetor is unlikely to include any information on the 
number and arrangement of cylinders in engines in which the 
carburetor may be used. Person's of ordinary skill in the 
art recognize the ordinary uses of components or sub- 
systems and it is impractical to cover the details and 
numerous variations of the larger systems. Neural networks 
have been under development since that the 1970 's and known 
uses exist in various fields including control systems and 
pattern recognition (e.g., facial recognition). Thus, the 
applicant has not described in detail larger systems in 
which the applicant's claimed neural networks can be 
incorporated. 

Examiner responds: 

Examiner disagrees. Examiner considers that it is still 

the direction of The Office to find a claim to be directed 

to one of the statutory classes and, if it is a process 

claim, to recite more than just a 101 judicial exception 

and have practical application. Examiner finds no 

practical application of Applicants' invention to be 

disclosed. Examiner considers Applicant's disclosure to 

provide only a demonstration of a number of different 

optimization algorithms that use derivative evaluation (see 

Specification, p. 29 of 59) to neural network training. 

Such results are considered to be directed to solution of 

problems in optimization theory and to thus be theoretical, 

rather than specific and credible real-world results (e.g., 

the final share price of State Street) . 
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Correspondence Information 

Any inquiry concerning this communication or earlier 

communications from the examiner should be directed to Nathan H. 

Brown, Jr. whose telephone number is 571-272- 8632. The 

examiner can normally be reached on M-F 0830-1700. If attempts 

to reach the examiner by telephone are unsuccessful, the 

examiner's supervisor, David Vincent can be reached on 571-272- 

3080. The fax phone number for the organization where this 

application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be 

obtained from the Patent Application Information Retrieval 

(PAIR) system. Status information for published applications 

may be obtained from either Private PAIR or Public PAIR. Status 

information for unpublished applications is available through 

Private PAIR only. For more information about the PAIR system, 

see http://pair-direct.uspto.gov. Should you have questions on 

access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free) . 

Nathan H. Brown, Jr. 
August 21, 2008 

/David R Vincent/ 

Supervisory Patent Examiner, Art Unit 2129 



